Image correction using a microlens array as a unit

ABSTRACT

A system constructs a composite image using focus assessment information of image regions.

This is a divisional application under 37 CFR 1.53(b), of pending priorapplication Ser. No. 10/764,340, filed on 21 Jan. 2004, entitled “IMAGECORRECTION USING A MICROLENS ARRAY AS A UNIT.”

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to, claims the earliest availableeffective filing date(s) from the following listed application(s) (the“Related Applications”) (e.g., claims earliest available priority datesfor other than provisional patent applications or claims benefits under35 USC § 119(e) for provisional patent applications), and incorporatesby reference in its entirety all subject matter of the following listedapplications; the present application also claims the earliest availableeffective filing date(s) from, and also incorporates by reference in itsentirety all subject matter of any and all parent, grandparent,great-grandparent, etc. applications of the Related Application(s).

-   1. The present application constitutes a division of United States    patent application entitled IMAGE CORRECTION USING A MICROLENS ARRAY    AS A UNIT, U.S. application Ser. No. 10/764,340, naming W. Daniel    Hillis; Nathan P. Myhrvold; and Lowell L. Wood, Jr. as inventors,    filed 21 Jan. 2004, which is currently co-pending, or is an    application of which a currently co-pending application is entitled    to the benefit of the filing date.-   2. For purposes of the USPTO extra-statutory requirements, the    present application constitutes a continuation-in-part of United    States patent application entitled LENS DEFECT CORRECTION, U.S.    application Ser. No. 10/738,626, naming William D. Hillis, Nathan P.    Myhrvold, and Lowell L. Wood Jr. as inventors, filed 16 Dec. 2003    which is currently co-pending, or is an application of which a    currently co-pending application is entitled to the benefit of the    filing date.-   3. For purposes of the USPTO extra-statutory requirements, the    present application constitutes a continuation-in-part of United    States patent application entitled IMAGE CORRECTION USING INDIVIDUAL    MANIPULATION OF MICROLENSES IN A MICROLENS ARRAY, naming William D.    Hillis, Nathan P. Myhrvold, and Lowell L. Wood Jr. as inventors,    filed 21 Jan. 2004 with U.S. application Ser. No. 10/764,431 which    is currently co-pending, or is an application of which a currently    co-pending application is entitled to the benefit of the filing    date.

The United States Patent Office (USPTO) has published a notice to theeffect that the USPTO's computer programs require that patent applicantsreference both a serial number and indicate whether an application is acontinuation or continuation-in-part. Stephen G. Kunin, Benefit ofPrior-Filed Application, USPTO Official Gazette Mar. 18, 2003, availableat http://www.uspto.gov/web/offices/com/sol/og/2003/week11 /patbene.htm.The present Applicant Entity (hereinafter “Applicant”) has providedabove a specific reference to the application(s)from which priority isbeing claimed as recited by statute. Applicant understands that thestatute is unambiguous in its specific reference language and does notrequire either a serial number or any characterization, such as“continuation” or “continuation-in-part,” for claiming priority to U.S.patent applications. Notwithstanding the foregoing, Applicantunderstands that the USPTO's computer programs have certain data entryrequirements, and hence Applicant is designating the present applicationas a continuation-in-part of its parent applications as set forth above,but expressly points out that such designations are not to be construedin any way as any type of commentary and/or admission as to whether ornot the present application contains any new matter in addition to thematter of its parent application(s).

All subject matter of the Related Applications and of any and allparent, grandparent, great-grandparent, etc. applications of the RelatedApplications is incorporated herein by reference to the extent suchsubject matter is not inconsistent herewith.

TECHNICAL FIELD

The present application relates, in general, to imaging.

SUMMARY

In one aspect, a method includes but is not limited to: capturing aprimary image with a microlens array at a primary position, themicrolens array having at least one microlens deviation that exceeds afirst tolerance from a target optical property; determining at least oneout-of-focus region of the primary image; capturing another image withthe microlens array at another position; determining a focus of at leastone region of the other image relative to a focus of the at least oneout-of-focus region of the primary image; and constructing a compositeimage in response to the at least one region of the other image having asharper focus relative to the focus of the at least one out-of-focusregion of the primary image. In addition to the foregoing, other methodaspects are described in the claims, drawings, and text forming a partof the present application

In one or more various aspects, related systems include but are notlimited to machinery and/or circuitry and/or programming for effectingthe herein-referenced method aspects; the machinery and/or circuitryand/or programming can be virtually any combination of hardware,software, and/or firmware configured to effect the foregoing-referencedmethod aspects depending upon the design choices of the system designer.

In one aspect, a system includes but is not limited to: a microlensarray having at least one microlens deviation that exceeds a firsttolerance from a target optical property; means for capturing a primaryimage with a lens at a primary position; means for determining at leastone out-of-focus region of the primary image; means for capturinganother image with the lens at another position; means for determining afocus of at least one region of the other image relative to a focus ofthe at least one out-of-focus region of the primary image; and means forconstructing a composite image in response to the at least one region ofthe other image having a sharper focus relative to the focus of the atleast one out-of-focus region of the primary image. In addition to theforegoing, other system aspects are described in the claims, drawings,and text forming a part of the present application.

In one aspect, a system includes but is not limited to: a microlensarray having at least one microlens deviation that exceeds a firsttolerance from a target optical property; an electro-mechanical systemconfigurable to capture a primary image with the microlens array at aprimary position said electro-mechanical system including at least oneof electrical circuitry operably coupled with a transducer, electricalcircuitry having at least one discrete electrical circuit, electricalcircuitry having at least one integrated circuit, electrical circuitryhaving at least one application specific integrated circuit, electricalcircuitry having a general purpose computing device configured by acomputer program, electrical circuitry having a memory device, andelectrical circuitry having a communications device; anelectro-mechanical system configurable to capture another image with themicrolens array at another position said electro-mechanical systemincluding at least one of electrical circuitry operably coupled with atransducer, electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice, and electrical circuitry having a communications device; anelectro-mechanical system configurable to determine at least oneout-of-focus region of the primary image said electro-mechanical systemincluding at least one of electrical circuitry operably coupled with atransducer, electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice, and electrical circuitry having a communications device; anelectro-mechanical system configurable to determine a focus of at leastone region of the other image relative to a focus of the at least oneout-of-focus region of the primary image said electro-mechanical systemincluding at least one of electrical circuitry operably-coupled with atransducer, electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice, and electrical circuitry having a communications device; anelectro-mechanical system configurable to determine a focus of at leastone region of the other image relative to a focus of the at least oneout-of-focus region of the primary image said electro-mechanical systemincluding at least one of electrical circuitry operably coupled with atransducer, electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice, and electrical circuitry having a communications device; and anelectro-mechanical system configurable to construct a composite image inresponse to the at least one region of the other image having a sharperfocus relative to the focus of the at least one out-of-focus region ofthe primary image said electro-mechanical system including at least oneof electrical circuitry operably coupled with a transducer, electricalcircuitry having at least one discrete electrical circuit, electricalcircuitry having at least one integrated circuit, electrical circuitryhaving at least one application specific integrated circuit, electricalcircuitry having a general purpose computing device configured by acomputer program, electrical circuitry having a memory device, andelectrical circuitry having a communications device. In addition to theforegoing, other system aspects are described in the claims, drawings,and text forming a part of the present application.

In one aspect, a method includes but is not limited to: capturing aprimary image with a microlens array at a primary position, saidcapturing effected with a photo-detector array having an imaging surfacedeviation that exceeds a first tolerance from a target surface position;determining at least one out-of-focus region of the primary image;capturing another image with the microlens array at another position;determining a focus of at least one region of the other image relativeto a focus of the at least one out-of-focus region of the primary image;and constructing a composite image in response to the at least oneregion of the other image having a sharper focus relative to the focusof the at least one out-of-focus region of the primary image. Inaddition to the foregoing, other method aspects are described in theclaims, drawings, and text forming a part of the present application.

In addition to the foregoing, various other method and or systemembodiments are set forth and described in the text (e.g., claims and/ordetailed description) and/or drawings of the present application.

The foregoing is a summary and thus contains, by necessity;simplifications, generalizations and omissions of detail; consequently,those skilled in the art will appreciate that the summary isillustrative only and is NOT intended to be in any way limiting. Otheraspects, inventive features, and advantages of the devices and/orprocesses described herein, as defined solely by the claims, will becomeapparent in the non-limiting detailed description set forth herein.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a front-plan view of image 100 of a person (e.g., person202 of FIG. 2) projected onto photo-detector array 102.

FIG. 2 depicts a side-plan view of lens system 200 that can give rise toimage 100 of FIG. 1.

FIG. 3 depicts a high level logic flowchart of a process.

FIG. 4 depicts a side-plan view of the system of FIG. 2 whereinmicrolens array 204 has been moved in accordance with aspects of theprocess shown and described in relation to FIG. 3.

FIG. 5 illustrates another side-plan view of the system of FIG. 2wherein microlens array 204 has been moved in accordance with aspects ofthe process shown and described in relation to FIG. 3.

The use of the same symbols in different drawings typically indicatessimilar or identical items.

DETAILED DESCRIPTION

With reference to the figures, and with reference now to FIG. 1, shownis a front-plan view of image 100 of a person (e.g., person 202 of FIG.2) projected onto photo-detector array 102. Image 100 is shown asdistorted due to defects in a microlens array through which image 100has been projected (e.g., microlens array 204 of lens system 200 of FIG.2). First portion 104 of image 100 is illustrated as large and blurry,which can occur when a microlens deviation causes first portion 104 ofimage 100 to come to a focus in front of an imaging surface ofphoto-detector array 102. Second, third, and fourth portions 106 ofimage 100 are illustrated as right sized, which can occur whenmicrolenses of the microlens array cause portions 106 to correctly focuson an imaging surface of photo-detector array 102. Fifth portion 108 ofimage 100 is shown as small and faint, which can occur when a microlensdeviation causes fifth portion 108 to come to a focus (virtual) behindan imaging surface of photo-detector array 102. In addition, althoughnot expressly shown, those having skill in the art will appreciate thatvarious microlens defects could also cause the image to be distorted inx-y; those having skill in the art will also appreciate that differentcolored wavelengths of light can in and of themselves focus at differentpositions due to differences in refraction of the different coloredwavelengths of light. In addition, although not expressly shown herein,those having skill in the art will appreciate that the subject matterdisclosed herein may serve to remedy misfocusings/distortions arisingfrom defects other than lens defects, such as, for example, defects inthe imaging surface of photo-detector array 102 and/or defects in framesthat hold microlens arrays.

Referring now to FIG. 2, depicted is a side-plan view of lens system 200that can give rise to image 100 of FIG. 1. Microlens array 204 of lenssystem 200 is illustrated as located at a primary position and havingmicrolens deviations that give rise to the five different portions ofimage 100 shown and described in relation to FIG. 1. First portion 104of image 100 is illustrated as misfocused in front of an imaging surfaceof photo-detector array 102, where the misfocus is due to a deviation ofmicrolens 252. Second, third, and fourth portions 106 of image 100 areillustrated as respectively right sized and focused by microlenses 250,254, and 258 on an imaging surface of photo-detector array 102. (It isrecognized that in side plan view the head and feet of person 202 wouldappear as lines; however, for sake of clarity they are shown in profilein FIG. 2 to help orient the reader relative to FIG. 1.) Fifth portion108 is shown as small and faint, and virtually misfocused behind animaging surface of photo-detector array 102, where the misfocus is dueto a deviation of microlens 256. In addition, although not expresslyshown herein, those having skill in the art will appreciate that thesubject matter of FIG. 2 is also illustrative of those situations inwhich one or more individual photo-detectors forming part of the imagingsurface of photo-detector array 102—rather than one or more microlensesof microlens array 204—deviate from one or more predefined positions byamounts such that image misfocuses/distortions arising from suchdeviations are unacceptable. That is, insofar as image misfocusingand/or distortion could just as easily arise from photo-detector array102 having mispositioned photo-detectors as from microlens array 204having mispositioned/defective lenses, the subject matter disclosedherein may serve to remedy misfocusings/distortions arising from defectsin the imaging surface of photo-detector array 102.

Continuing to refer to FIG. 2, further shown are components that canserve as an environment for the process shown and described in relationto FIG. 3. Specifically, controller 208 is depicted as controlling theposition of microlens array 204 of lens system 200 (e.g., via use of afeedback control subsystem). Image capture unit 206 is illustrated asreceiving image data from photo-detector array 102 and receiving controlsignals from controller 208. Image capture unit 206 is shown astransmitting captured image information to focus detection unit 210.Focus detection unit 210 is depicted as transmitting focus data to imageconstruction unit 212. Image construction unit 212 is illustrated astransmitting a composite image to image store/display unit 214.

With reference now to FIG. 3, depicted is a high level logic flowchartof a process. Method step 300 shows the start of the process. Methodstep 302 depicts capturing a primary image with a microlens array at aprimary position, the microlens array having at least one microlensdeviation that exceeds a first tolerance from a target optical property.Examples of the array having at least one microlens deviation thatexceeds a first tolerance from a target optical property include (a)where at least one microlens position exceeds a first tolerance from atleast one defined microlens position, and (b) where at least onemicrolens of the microlens array has at least one focal length thatexceeds a first tolerance from a defined focal length (e.g., a microlensdeviation that would produce portion 108 of image 100 at some placebehind an imaging surface of photo-detector array 102 or a microlensdeviation that would produce portion 104 at some place in front of theimaging surface of photo-detector array 102 where the distance in frontor back of the imaging surface exceeds a defined tolerance distancewhere an image captured with the photo-detector array 102 is deemedacceptable). Specific instances of the foregoing include a microlens ofthe microlens array having at least one spherical aberration thatexceeds a first tolerance from a defined spherical aberration, and amicrolens of the microlens array having at least one cylindricalaberration that exceeds a first tolerance from a defined cylindricalaberration. Alternatively, the microlens array may have some combinationof microlenses having such defects. In one implementation, method step302 includes the sub-step of capturing the primary image at an averageprimary focal surface location of the microlens array (e.g., a definedfocal surface of the microlens array where an image would form if themicrolens array had no microlenses having aberrations outside aspecified tolerance). In another implementation, method step 302includes the sub-step of capturing the primary image with aphoto-detector array at the average primary focal surface location ofthe microlens array (e.g., positioning the microlens array such that adefined focal surface of the lens coincides with an imaging surface of aphoto-detector array).

Referring again to FIG. 2, one specific example of method step 302 (FIG.3) would be controller 208 directing lens system 200 to positionmicrolens array 204 at a primary position, and thereafter instructingimage capture unit 206 to capture an image from photo-detector array102.

With reference again to FIG. 3, method step 304 illustrates determiningat least one out-of-focus region of the primary image (or determining atleast one focused region of the primary image). In one implementation,method step 304 includes the sub-step of calculating a Fourier transformof at least a part of the primary image (e.g., sharp, or in-focus imagesproduce abrupt transitions that often have significant high frequencycomponents).

Referring again to FIG. 2, one specific example of method step 304 (FIG.3) would be focus detection unit 210 performing a Fourier transform andsubsequent analysis on at least a part of an image captured by imagecapture unit 206 when lens 204 was at the primary position. In thisexample, focus detection unit 210 could deem portions of the imagehaving significant high frequency components as “in focus” images. As amore specific example, the Fourier transform and analysis may beperformed on one or more parts of the image that are associated with oneor more microlenses 250-258 of microlens array 204.

With reference again to FIG. 3, method step 306 shows capturing anotherimage with the microlens array at another position. In oneimplementation, method step 306 includes the sub-step of capturing theother image at the average primary focal surface location of themicrolens array at the primary position. In another implementation, thestep of capturing the other image at a primary focal surface location ofthe microlens array at the primary position further includes thesub-step of moving at least a part of the microlens array to the otherposition; and capturing the other image with a photo-detector array atthe primary focal surface location of the microlens at the primaryposition (e.g., microlens array 204 is moved to another position, whilephoto-detector array 102 remains stationary, such as shown and describedin relation to FIGS. 4 and 5). In another implementation, the step ofmoving at least a part of the microlens array to the other positionfurther includes the sub-step of moving the at least a part of themicrolens array to the other position within at least one distanceconstrained by a predefined variation from at least one definedmicrolens position. In another implementation, the step of moving atleast a part of the microlens array to the other position furtherincludes the sub-step of moving an intermediary lens. In anotherimplementation, the step of moving at least a part of the microlensarray to the other position further includes the sub-step of distortingthe microlens array such that the at least a part of the microlens arrayresides at the other position (e.g., a part of microlens array 204 ismoved to another position, such as might happen if microlens array 204were to be compressed laterally in a controlled manner, whilephoto-detector array 102 remains stationary, such as shown and describedin relation to FIGS. 4 and 5).

Referring now to FIGS. 2, 4 and/or 5, one specific example of methodstep 306 (FIG. 3) would be controller 208 directing lens system 200 toposition microlens array 204 at a position other than the primaryposition and thereafter instructing image capture unit 206 to capture animage from photo-detector array 102. FIG. 4 shows and describes movingat least a portion of microlens array 204 forward of the primaryposition (e.g., such as by controller 208 moving microlens array 204forward, or causing microlens array 204 to be compressed such that apart of microlens array 204 moves forward relative to an imaging surfaceof photo-detector array 102). FIG. 5 shows and describes moving at leasta portion of the microlens array rearward of the primary position (e.g.,such as by controller 208 moving microlens array 204 rearward, orcausing microlens array 204 to be compressed such that a part ofmicrolens array 204 moves rearward relative to an imaging surface ofphoto-detector array 102).

With reference again to FIG. 3, method step 308 depicts determining afocus of at least one region of the other image relative to a focus ofthe at least one out-of-focus region of the primary image. In oneimplementation, method step 308 includes the sub-step of calculating aFourier transform of at least a part of at least one region of the otherimage (e.g., sharp or in-focus images produce abrupt transitions thatoften have significant high frequency components). In oneimplementation, the step of calculating a Fourier transform of at leasta part of at least one region of the other image (e.g., sharp orin-focus images produce abrupt transitions that often have significanthigh frequency components) includes the sub-step of mapping at least oneregion of the primary image with at least one region of the other image(e.g., mapping an out-of-focus region of the first image to acorresponding region of the second image). As a more specific example,the Fourier transform and analysis may be performed on one or more partsof the image that are associated with one or more microlenses of themicrolens array (e.g., mapping at least one region of the primary imageassociated with at least one specific microlens against the at least oneregion of the other image associated with the at least one specificmicrolens).

Referring again to FIGS. 2, 4 and/or 5, one specific example of methodstep 308 (FIG. 3) would be focus detection unit 210 performing a Fouriertransform and subsequent analysis on at least a part of an imagecaptured by image capture unit 206 when microlens array 204 was at theother position specified by controller 208.

With reference again to FIG. 3, method step 310 depicts constructing acomposite image in response to the at least one region of the otherimage having a sharper focus relative to the focus of the at least oneout-of-focus region of the primary image. In one implementation, thestep of constructing a composite image in response to the at least oneregion of the other image having a sharper focus relative to the focusof the at least one out-of-focus region of the primary image includesthe sub-step of replacing at least a part of the out-of-focus region ofthe primary image with at least a part of the at least one region of theother image. In yet another implementation, the step of constructing acomposite image in response to the at least one region of the otherimage having a sharper focus relative to the focus of the at least oneout-of-focus region of the primary image includes the sub-step ofutilizing at least one of tiling image processing techniques, morphingimage processing techniques, blending image processing techniques, andstitching image processing techniques.

In yet another implementation, the step of constructing a compositeimage in response to the at least one region of the other image having asharper focus relative to the focus of the at least one out-of-focusregion of the primary image includes the sub-steps of correlating afeature of the primary image with a feature of the other image;detecting at least one of size, color, and displacement distortion of atleast one of the primary image and the other image; correcting thedetected at least one of size, color, and displacement distortion of theat least one of the primary image and the other image; and assemblingthe composite image using the corrected distortion. In yet anotherimplementation, the step of constructing a composite image in responseto the at least one region of the other image having a sharper focusrelative to the focus of the at least one out-of-focus region of theprimary image includes the sub-step of correcting for motion between theprimary and the other image.

Referring again to FIGS. 2, 4 and/or 5, one specific example of methodstep 310 (FIG. 3) would be image construction unit 212 creating acomposite image by replacing those portions of an image of person 202captured at a primary position with more in-focus portions of an imageof person 202 captured by image capture unit 206 when microlens array204 was at the other position. In one implementation of the example,image construction unit 212 corrects for the motion between images usingconventional techniques if such correction is desired. In anotherimplementation of the example, motion correction is not used.

With reference again to FIG. 3, method step 312 shows a determination ofwhether an aggregate change in position, relative to the primaryposition of method step 302, has exceeded a maximum expected deviationof the microlens array. For example, even with a relatively poor qualitymicrolens array, there will typically be an upper manufacturing limitbeyond which microlens deviations are not expected to go (e.g., themicrolens array has manufacturing criteria such that each microlens inthe array provide a focal length of 5 mm+/−0.05 mm).

Referring again to FIGS. 2, 4 and/or 5, one specific example of methodstep 312. (FIG. 3) would be controller 208 comparing an aggregatemovement in a defined direction against a pre-stored upper limitdeviation value. In an implementation of the example illustrated in FIG.4, if microlens array 204 has manufacturing criteria such as a focallength of 5 mm+/−0.05 mm, controller 208 will determine whether thetotal forward movement of the microlens array is greater than 0.05 mmrelative to the primary position. In an implementation of the exampleillustrated in FIG. 5, if microlens array 204 has manufacturing criteriasuch as a focal length of 5 mm+/−0.05 mm, controller 208 will determinewhether the total rearward movement of microlens array 204 is greaterthan 0.05 mm relative to the primary position.

With reference again to FIG. 3, if the inquiry of method step 312 yieldsa determination that the aggregate change in position has met orexceeded the maximum expected deviation of the microlens array, theprocess proceeds to method step 314. Method step 314 illustrates thatthe current composite image (e.g., of method step 310) is stored and/ordisplayed. One specific example of method step 314 would be imagestore/display unit 214 either storing or displaying the composite image.

Method step 316 shows the end of the process.

Returning to method step 312, shown is that in the event that the upperlimit on microlens array tolerance has not been met or exceeded, theprocess proceeds to method step 306 and continues as described herein.

Referring now to FIG. 4, depicted is a side-plan view of the system ofFIG. 2 wherein microlens array 204 has been moved in accordance withaspects of the process shown and described in relation to FIG. 3.Microlens array 204 of lens system 200 is illustrated as having beenmoved to another position forward of the primary position which gaverise to the five different portions of image 100 shown and described inrelation to FIGS. 1 and 2. Specifically, microlens array 204 of lenssystem 200 is illustrated as repositioned such that fifth portion 108 ofimage 100 is right sized and focused on an imaging surface ofphoto-detector array 102 (e.g., as shown and described in relation tomethod step 306). In one implementation, fifth portion 108 of image 100can be combined with previously captured in focus and right sizedportions 106 (e.g., FIGS. 1 and 2) to create a composite image such thatthe defects associated with fifth portion 108 as shown and described inrelation to FIGS. 1 and 2 are alleviated (e.g., as shown and describedin relation to method step 310). The remaining components and controlaspects of the various parts of FIG. 4 function as described elsewhereherein.

With reference now to FIG. 5, illustrated is another side-plan view ofthe system of FIG. 2 wherein microlens array 204 has been moved inaccordance with aspects of the process shown and described in relationto FIG. 3. Microlens array 204 of lens system 200 is illustrated ashaving been moved to another position rearward of the primary positionwhich gave rise to the five different portions of image 100 shown anddescribed in relation to FIG. 1. Specifically, microlens array 204 oflens system 200 is illustrated as positioned such that first portion 104of image 100 is right sized and focused on an imaging surface ofphoto-detector array 102 (e.g., as described in relation to method step306). In one implementation, first portion 104 of image 100 can becombined with previously captured in focus and right sized portions 106,108 (e.g., FIGS. 1, 2, and 4) to create a composite image such that thedefects associated with first portion 104 as shown and described inrelation to FIGS. 1 and 2 are alleviated (e.g., as shown and describedin relation to method step 310). The remaining components and controlaspects of the various parts of FIG. 5 function as described elsewhereherein.

Those having skill in the art will recognize that the state of the arthas progressed to the point where there is little distinction leftbetween hardware and software implementations of aspects of systems; theuse of hardware or software is generally (but not always, in that incertain contexts the choice between hardware and software can becomesignificant) a design choice representing cost vs. efficiency tradeoffs.Those having skill in the art will appreciate that there are variousvehicles by which processes and/or systems described herein can beeffected (e.g., hardware, software, and/or firmware), and that thepreferred vehicle will vary with the context in which the processes aredeployed. For example, if an implementer determines that speed andaccuracy are paramount, the implementer may opt for a hardware and/orfirmware vehicle; alternatively, if flexibility is paramount, theimplementer may opt for a solely software implementation; or, yet againalternatively, the implementer may opt for some combination of hardware,software, and/or firmware. Hence, there are several possible vehicles bywhich the processes described herein may be effected, none of which isinherently superior to the other in that any vehicle to be utilized is achoice dependent upon the context in which the vehicle will be deployedand the specific concerns (e.g., speed, flexibility, or predictability)of the implementer, any of which may vary. Those skilled in the art willrecognize that optical aspects of implementations will requireoptically-oriented hardware, software, and or firmware.

The foregoing detailed description has set forth various embodiments ofthe devices and/or processes via the use of block diagrams, flowcharts,and examples. Insofar as such block diagrams, flowcharts, and examplescontain one or more functions and/or operations, it will be understoodas notorious by those within the art that each function and/or operationwithin such block diagrams, flowcharts, or examples can be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment, thepresent invention may be implemented via Application Specific IntegratedCircuits (ASICs), Field Programmable Gate Arrays (FPGAs), or otherintegrated formats. However, those skilled in the art will recognizethat the embodiments disclosed herein, in whole or in part, can beequivalently implemented in standard integrated circuits, as one or morecomputer programs running on one or more computers (e.g., as one or moreprograms running on one or more computer systems), as one or moreprograms running on one or more processors (e.g., as one or moreprograms running on one or more microprocessors), as firmware, or asvirtually any combination thereof, and that designing the circuitryand/or writing the code for the software and or firmware would be wellwithin the skill of one of skill in the art in light of this disclosure.In addition, those skilled in the art -will appreciate that themechanisms of the present invention are capable of being distributed asa program product in a variety of forms, and that an illustrativeembodiment of the present invention applies equally regardless of theparticular type of signal bearing media used to actually carry out thedistribution. Examples of a signal bearing media include, but are notlimited to, the following: recordable type media such as floppy disks,hard disk drives, CD ROMs, digital tape, and computer memory; andtransmission type media such as digital and analog communication linksusing TDM or IP based communication links (e.g., packet links).

In a general sense, those skilled in the art will recognize that thevarious embodiments described herein which can be implemented,individually and/or collectively, by various types of electro-mechanicalsystems having a wide range of electrical components such as hardware,software, firmware, or virtually any combination thereof, and a widerange of components that may impart mechanical force or motion such asrigid bodies, spring or torsional bodies, hydraulics, andelectro-magnetically actuated devices, or virtually any combinationthereof. Consequently, as used herein “electro-mechanical system”includes, but is not limited to, electrical circuitry operably coupledwith a transducer (e.g., an actuator, a motor, a piezoelectric crystal,etc.), electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry forming a general purpose computing deviceconfigured by a computer program (e.g., a general purpose computerconfigured by a computer program which at least partially carries outprocesses and/or devices described herein, or a microprocessorconfigured by a computer program which at least partially carries outprocesses and/or devices described herein), electrical circuitry forminga memory device (e.g., forms of random access memory), electricalcircuitry forming a communications device (e.g., a modem, communicationsswitch, or optical-electrical equipment), and any non-electrical analogthereto, such as optical or other analogs. Those skilled in the art willalso appreciate that examples of electro-mechanical systems include butare not limited to a variety of consumer electronics systems, as well asother systems such as motorized transport systems, factory automationsystems, security systems, and communication/computing systems. Thoseskilled in the art will recognize that electro-mechanical as used hereinis not necessarily limited to a system that has both electrical andmechanical actuation except as context may dictate otherwise.

Those skilled in the art will recognize that it is common within the artto describe devices and/or processes in the fashion set forth herein,and thereafter use standard engineering practices to integrate suchdescribed devices and/or processes into image processing systems. Thatis, at least a portion of the devices and/or processes described hereincan be integrated into an image processing system via a reasonableamount of experimentation. Those having skill in the art will recognizethat a typical image processing system generally includes one or more ofa system unit housing, a video display device, a memory such as volatileand non-volatile memory, processors such as microprocessors and digitalsignal processors, computational entities such as operating systems,drivers, and applications programs, one or more interaction devices,such as a touch pad or screen, control systems including feedback loopsand control motors (e.g., feedback for sensing lens position and/orvelocity; control motors for moving/distorting lenses to give desiredfocuses. A typical image processing system may be implemented utilizingany suitable commercially available components, such as those typicallyfound in digital still systems and/or digital motion systems.

The foregoing described embodiments depict different componentscontained within, or connected with, different other components. It isto be understood that such depicted architectures are merely exemplary,and that in fact many other architectures can be implemented whichachieve the same functionality. In a conceptual sense, any arrangementof components to achieve the same functionality is effectively“associated” such that the desired functionality is achieved. Hence, anytwo components herein combined to achieve a particular functionality canbe seen as “associated with” each other such that the desiredfunctionality is achieved, irrespective of architectures or intermedialcomponents. Likewise, any two components so associated can also beviewed as being “operably connected” or “operably coupled” to each otherto achieve the desired functionality.

While particular embodiments of the present invention have been shownand described, it will be understood by those skilled in the art that,based upon the teachings herein, changes and modifications may be madewithout departing from this invention and its broader aspects and,therefore, the appended claims are to encompass within their scope allsuch changes and modifications as are within the true spirit and scopeof this invention. Furthermore, it is to be understood that theinvention is solely defined by the appended claims. It will beunderstood by those within the art that, in general, terms used herein,and especially in the appended claims (e.g., bodies of the appendedclaims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”“comprise” and variations thereof, such as, “comprises” and “comprising”are to be construed in an open, inclusive sense, that is as “including,but not limited to,” etc.). It will be further understood by thosewithin the art that if a specific number of an introduced claimrecitation is intended, such an intent will be explicitly recited in theclaim, and in the absence of such recitation no such intent is present.For example, as an aid to understanding, the following appended claimsmay contain usage of the introductory phrases “at least one” and “one ormore” to introduce claim recitations. However, the use of such phrasesshould not be construed to imply that the introduction of a claimrecitation by the indefinite articles “a” or “an” limits any particularclaim containing such introduced claim recitation to inventionscontaining only one such recitation, even when the same claim includesthe introductory phrases “one or more” or “at least one” and indefinitearticles such as “a” or “an” (e.g., “a” and/or “an” should typically beinterpreted to mean “at least one” or “one or more”); the same holdstrue for the use of definite articles used to introduce claimrecitations. In addition, even if a specific number of an introducedclaim recitation is explicitly recited, those skilled in the art willrecognize that such recitation should typically be interpreted to meanat least the recited number (e.g., the bare recitation of “tworecitations,” without other modifiers, typically means at least tworecitations, or two or more recitations).

1. A system comprising: a photo-detector array; a microlens array havingat least one microlens deviation that exceeds a first tolerance from atarget optical property; a controller configured to position saidmicrolens array at a primary and another position relative to saidphoto-detector array and to cause an image capture signal at the primaryand the other position; and an image construction unit configured toreplace at least one out-of-focus region of a first image captured atthe primary position with a more in-focus region of another imagecaptured at the other position.
 2. The system of claim 1, wherein saidcontroller configured to position said microlens array at a primary andanother position relative to said photo-detector array and to cause animage capture signal at the primary and the other position furthercomprises: a transducer system having a control signal input operablycoupled with said controller and a motion output operably coupled withsaid microlens array.
 3. The system of claim 2, wherein said transducersystem further comprises: an electric motor operably coupled to movesaid microlens array.
 4. The system of claim 2, wherein said transducersystem further comprises: an electric motor operably coupled to distortsaid microlens array.
 5. The system of claim 1, wherein said imageconstruction unit configured to replace at least one out-of-focus regionof a first image captured at the primary position with a more in-focusregion of another image captured at the other position furthercomprises: circuitry for constructing at least one out-of-focus regionof a first image captured at the primary position with a more in-focusregion of another image captured at the other position said circuitryincluding at least one of electrical circuitry having at least onediscrete electrical circuit, electrical circuitry having at least oneintegrated circuit, electrical circuitry having at least one applicationspecific integrated circuit, electrical circuitry having a generalpurpose computing device configured by a computer program, electricalcircuitry having a memory device, and/or electrical circuitry having acommunications device.
 6. The system of claim 1, further comprising: afocus detection unit having an input operably coupleable with an outputof an image capture unit, said focus detection unit further having anoutput operably couplable with said image construction unit.
 7. Thesystem of claim 6, wherein said focus detection unit further comprises:circuitry for determining the at least one out-of-focus region of thefirst image captured at the primary position; and circuitry fordetermining the more in-focus region of the other image captured at theother position.
 8. The system of claim 7, wherein said circuitry fordetermining the more in-focus region the other image captured at theother position further comprises: circuitry for mapping the at least oneout-of-focus region of the first image captured at the primary positionwith the more in-focus region of the other image captured at the otherposition.
 9. The system of claim 1, wherein said image construction unitconfigured to replace at least one out-of-focus region of a first imagecaptured at the primary position with a more in-focus region of anotherimage captured at the other position further comprises: circuitry forreplacing at least a part of the out-of-focus region of the first imagecaptured at the primary position with at least a part of the morein-focus region of the other image captured at the other position. 10.The system of claim 1, wherein said image construction unit configuredto replace at least one out-of-focus region of a first image captured atthe primary position with a more in-focus region of another imagecaptured at the other position further comprises: circuitry fordetecting at least one of size, color, and/or displacement distortion ofat least one of the first image captured at the primary position and theother image captured at the other position.
 11. The system of claim 1,wherein said image construction unit configured to replace at least oneout-of-focus region of a first image captured at the primary positionwith a more in-focus region of another image captured at the otherposition further comprises: circuitry for correcting at least one ofsize, color, displacement, and/or motion distortion of at least one ofthe first image captured at the primary position and the other imagecaptured at the other position.
 12. The system of claim 1, wherein saidimage construction unit configured to replace at least one out-of-focusregion of a first image captured at the primary position with a morein-focus region of another image captured at the other position furthercomprises: circuitry for constructing an image utilizing at least one ofsize, color, displacement, and/or motion distortion of at least one ofthe first image captured at the primary position and/or the other imagecaptured at the other position.
 13. The system of claim 1, where saidcontroller configured to position said microlens array at a primary andanother position relative to said photo-detector array and to cause animage capture signal at the primary and the other position furthercomprises: circuitry for directing a transducer system to move themicrolens array to the primary position; circuitry for directing animage capture unit to capture an image responsive to a notification thatthe microlens array has been moved to the primary posistion.
 14. Thesystem of claim 1, where said controller configured to position saidmicrolens array at a primary and another position relative to saidphoto-detector array and to cause an image capture signal at the primaryand the other position further comprises: circuitry for directing atransducer system to move the microlens array to the other position;circuitry for directing an image capture unit to capture an imageresponsive to a notification that the microlens array has been moved tothe other position.
 15. The system of claim 1, where said controllerconfigured to position said microlens array at a primary and anotherposition relative to said photo-detector array and to cause an imagecapture signal at the primary and the other position further comprises:circuitry for constraining movement of said microlens array by apredefined variation from at least one defined microlens position. 16.The system of claim 1, where said controller configured to position saidmicrolens array at a primary and another position relative to saidphoto-detector array and to cause an image capture signal at the primaryand the other position further comprises: a micro-electromechanicalsystem (MEMS) operably coupled with at least one microlens of saidmicrolens array.
 17. A system comprising: a microlens array having atleast one microlens deviation that exceeds a first tolerance from atarget optical property; an electro-mechanical system configurable tocapture a primary image with the microlens array at a primary positionsaid electro-mechanical system including at least one of electricalcircuitry operably coupled with a transducer, electrical circuitryhaving at least one discrete electrical circuit, electrical circuitryhaving at least one integrated circuit, electrical circuitry having atleast one application specific integrated circuit, electrical circuitryhaving a general purpose computing device configured by a computerprogram, electrical circuitry having a memory device, and/or electricalcircuitry having a communications device; an electro-mechanical systemconfigurable to capture another image with the microlens array atanother position said electro-mechanical system including at least oneof electrical circuitry operably coupled with a transducer, electricalcircuitry having at least one discrete electrical circuit, electricalcircuitry having at least one integrated circuit, electrical circuitryhaving at least one application specific integrated circuit, electricalcircuitry having a general purpose computing device configured by acomputer program, electrical circuitry having a memory device, and/orelectrical circuitry having a communications device; anelectro-mechanical system configurable to determine at least oneout-of-focus region of the primary image said electro-mechanical systemincluding at least one of electrical circuitry operably coupled with atransducer, electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice, and/or electrical circuitry having a communications device; anelectro-mechanical system configurable to determine a focus of at leastone region of the other image relative to a focus of the at least oneout-of-focus region of the primary image said electro-mechanical systemincluding at least one of electrical circuitry operably coupled with atransducer, electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice, and/or electrical circuitry having a communications device; anelectro-mechanical system configurable to determine a focus of at leastone region of the other image relative to a focus of the at least oneout-of-focus region of the primary image said electro-mechanical systemincluding at least one of electrical circuitry operably coupled with atransducer, electrical circuitry having at least one discrete electricalcircuit, electrical circuitry having at least one integrated circuit,electrical circuitry having at least one application specific integratedcircuit, electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice, and/or electrical circuitry having a communications device; andan electro-mechanical system configurable to construct a composite imagein response to the at least one region of the other image having asharper focus relative to the focus of the at least one out-of-focusregion of the primary image said electro-mechanical system including atleast one of electrical circuitry operably coupled with a transducer,electrical circuitry having at least one discrete electrical circuit,electrical circuitry having at least one integrated circuit, electricalcircuitry having at least one application specific integrated circuit,electrical circuitry having a general purpose computing deviceconfigured by a computer program, electrical circuitry having a memorydevice and/or electrical circuitry having a communications device.
 18. Amethod comprising: capturing a primary image with a microlens array at aprimary position, said capturing effected with a photo-detector arrayhaving an imaging surface deviation that exceeds a first tolerance froma target surface position; determining at least one out-of-focus regionof the primary image; capturing another image with the microlens arrayat another position; determining a focus of at least one region of theother image relative to a focus of the at least one out-of-focus regionof the primary image; and constructing a composite image in response tothe at least one region of the other image having a sharper focusrelative to the focus of the at least one out-of-focus region of theprimary image.
 19. The method of claim 18, wherein said capturingeffected with a photo-detector array having an imaging surface deviationthat exceeds a first tolerance from a target surface position furthercomprises: the photo-detector array having at least one photo-detectorposition that exceeds a first tolerance from at least one definedphoto-detector position.